Aligning physical elements with persons' attitude: an approach using Rasch measurement theory

Affective engineering uses mathematical models to convert the information obtained from persons' attitude to physical elements into an ergonomic design. However, applications in the domain have not in many cases met measurement assumptions. This paper proposes a novel approach based on Rasch measurement theory to overcome the problem. The research demonstrates that if data fit the model, further variables can be added to a scale. An empirical study was designed to determine the range of compliance where consumers could obtain an impression of a moisturizer cream when touching some product containers. Persons, variables and stimulus objects were parameterised independently on a linear continuum. The results showed that a calibrated scale preserves comparability although incorporating further variables

Using item response theory to explore the psychometric properties of extended matching questions examination in undergraduate medical education

BACKGROUND: As assessment has been shown to direct learning, it is critical that the examinations developed to test clinical competence in medical undergraduates are valid and reliable. The use of extended matching questions (EMQ) has been advocated to overcome some of the criticisms of using multiple-choice questions to test factual and applied knowledge. METHODS: We analysed the results from the Extended Matching Questions Examination taken by 4th year undergraduate medical students in the academic year 2001 to 2002. Rasch analysis was used to examine whether the set of questions used in the examination mapped on to a unidimensional scale, the degree of difficulty of questions within and between the various medical and surgical specialties and the pattern of responses within individual questions to assess the impact of the distractor options. RESULTS: Analysis of a subset of items and of the full examination demonstrated internal construct validity and the absence of bias on the majority of questions. Three main patterns of response selection were identified. CONCLUSION: Modern psychometric methods based upon the work of Rasch provide a useful approach to the calibration and analysis of EMQ undergraduate medical assessments. The approach allows for a formal test of the unidimensionality of the questions and thus the validity of the summed score. Given the metric calibration which follows fit to the model, it also allows for the establishment of items banks to facilitate continuity and equity in exam standards

Critical Values for Yen’s Q3: Identification of Local Dependence in the Rasch model using Residual Correlations

The assumption of local independence is central to all IRT models. Violations can lead to inflated estimates of reliability and problems with construct validity. For the most widely used fit statistic Q3 there are currently no well-documented suggestions of the critical values which should be used to indicate local dependence, and for this reason a variety of arbitrary rules of thumb are used. In this study, we used an empirical data example and Monte Carlo simulation to investigate the different factors that can influence the null distribution of residual correlations, with the objective of proposing guidelines that researchers and practitioners can follow when making decisions about local dependence during scale development and validation. We propose that a parametric bootstrapping procedure should be implemented in each separate situation in order to obtain the critical value of local dependence applicable to the data set, and provide example critical values for a number of data structure situations. The results show that for the Q3 fit statistic no single critical value is appropriate for all situations, as the percentiles in the empirical null distribution are influenced by the number of items, the sample size, and the number of response categories. Furthermore, our results show that local dependence should be considered relative to the average observed residual correlation, rather than to a uniform value, as this results in more stable percentiles for the null distribution of an adjusted fit statistic

Ezidi voices: The communication of COVID-19 information amongst a refugee community in rural Australia- a qualitative study

Background: There is growing evidence that government health information related to COVID-19 has failed to adequately reach culturally and linguistically diverse (CALD) populations in Australia. Refugees are a unique sub-set of the CALD communities and are subject to numerous barriers preventing adequate health care, both pre- and post-migration. The barriers are accentuated during emergencies, such as a pandemic, as a result of an intersection of various social and economic inequalities. The recently resettled Ezidi refugee community in a regional area of Australia is an example of a community sitting at the intersection of various inequities and thus at greater risk from COVID-19. The purpose of this study is to describe the experiences of the Ezidi in a regional area with COVID-19 information and how this has been communicated to and shared within this group; what barriers the community may experience in accessing COVID-19 information; and how the government-led COVID-19 information communication could be improved. Methods: This qualitative study was designed to explore the perceptions and views of the Ezidi and service providers regarding COVID-19 messaging. Multicultural and Refugee Health staff facilitated interviews with four local service providers and ten Ezidi community members, including seven influential leaders. Thematic analysis was employed across individual, pair and group data analysis. Similar categories were grouped into themes. Results: The main findings of the study are: the refugee experience influences the communication of COVID-19 messages; cultural, social and gender norms influence responses to COVID-19; trusted individuals and service providers are key in communities’ uptake of COVID-19 messages; currently available governmental COVID-19 information resources and sharing strategies were found unhelpful and inappropriate; COVID-19 communiqués and message delivery for this regional minority refugee community can be improved. Conclusion: The recently resettled Ezidi community, and likely other similar communities, would benefit from tailored engagement by government organisations, as well as settlement services to improve the communication of COVID-19 health information and reduce related inequities

Reliability and responsiveness of measures of pain in people with osteoarthritis of the knee: a psychometric evaluation

PURPOSE: To examine the fit between data from the Short Form McGill Pain Questionnaire (SF-MPQ-2) and the Rasch model, and to explore the reliability and internal responsiveness of measures of pain in people with knee osteoarthritis. METHODS: Participants with knee osteoarthritis completed the SF-MPQ-2, Intermittent and Constant Osteoarthritis Pain questionnaire (ICOAP) and painDETECT. Participants were sent the same questionnaires 3 and 6 months later. RESULTS: Fit to the Rasch model was not achieved for the SF-MPQ-2 Total scale. The Continuous subscale yielded adequate fit statistics after splitting item 10 on uniform DIF for gender, and removing item 9. The Intermittent subscale fit the Rasch model after rescoring items. The Neuropathic subscale had relatively good fit to the model. Test-retest reliability was satisfactory for most scales using both original and Rasch scoring ranging from fair to substantial. Effect sizes ranged from 0.13 to 1.79 indicating good internal responsiveness for most scales. CONCLUSIONS: These findings support the use of ICOAP subscales as reliable and responsive measure of pain in people with knee osteoarthritis. The MPQ-SF-2 subscales found to be acceptable alternatives. Implications for Rehabilitation The McGill Pain Questionnaire short version 2 is not a unidimensional scale in people with knee osteoarthritis, whereas three of the subscales are unidimensional. The McGill Pain Questionnaire short version 2 Affective subscale does not have good measurement properties for people with knee osteoarthritis. The McGill Pain Questionnaire short version 2 and the Intermittent and Constant Osteoarthritis Pain scales can be used to assess change over time. The painDETECT performs better as a screening measure than as an outcome measure

Factors contributing to the sharing of COVID-19 health information amongst refugee communities in a regional area of Australia: a qualitative study

Background: The COVID-19 pandemic has had a disproportionate impact on culturally and linguistically diverse (CALD) groups worldwide. Newly emerging CALD populations formed by recently arrived refugees are predisposed to even greater health disadvantages due to complexities of the refugee experience. The aim of this study was to explore how culture, refugee experiences and existing relationships shaped what COVID-19 messages were listened to and shared during the early-mid phases of the pandemic. The work focused on three newly emerging refugee groups in the Hunter New England region, Australia: Afghan, Congolese and Syrian communities. Methods: Qualitative, semi-structured interviews were conducted to explore the experiences and stories of 15 adult community members, nine influential members and six service providers. All community members arrived in Australia on or after January 2014. Interpreter-assisted interviews were conducted with small groups or individuals, audio-recorded and transcribed verbatim in English. Three levels of thematic data analysis were employed to uncover the important issues and experiences of the participants. Results: Three key themes and several subthemes were identified. The themes were: 1) Experience as a refugee uniquely influences COVID-19 message communication; 2) Refugee groups use diverse practices when accessing and sharing COVID-19 messages; and 3) Official government messages could be improved by listening and tailoring to community needs. Conclusions: Effective health messaging relies on reaching communities in a culturally acceptable and meaningful way. Official COVID-19 messages can be tailored to engage newly emerging communities by improving the quality of the content, delivery and format whilst working collaboratively with communities and trusted service providers. Further mutual research is needed to understand emerging communities’ viewpoints. The use of culturally informed approaches is recommended

Internal construct validity of the Warwick-Edinburgh Mental Well-being Scale (WEMWBS): a Rasch analysis using data from the Scottish Health Education Population Survey

Background: The Warwick-Edinburgh Mental Well-Being Scale (WEMWBS) was developed to meet demand for instruments to measure mental well-being. It comprises 14 positively phrased Likert-style items and fulfils classic criteria for scale development. We report here the internal construct validity of WEMWBS from the perspective of the Rasch measurement model. Methods: The model was applied to data collected from 779 respondents in Wave 12 (Autumn 2006) of the Scottish Health Education Population Survey. Respondents were aged 16–74 (average 41.9) yrs. Results: Initial fit to model expectations was poor. The items 'I've been feeling good about myself', 'I've been interested in new things' and 'I've been feeling cheerful' all showed significant misfit to model expectations, and were deleted. This led to a marginal improvement in fit to the model. After further analysis, more items were deleted and a strict unidimensional seven item scale (the Short Warwick Edinburgh Mental Well-Being Scale (SWEMWBS)) was resolved. Many items deleted because of misfit with model expectations showed considerable bias for gender. Two retained items also demonstrated bias for gender but, at the scale level, cancelled out. One further retained item 'I've been feeling optimistic about the future' showed bias for age. The correlation between the 14 item and 7 item versions was 0.954. Given fit to the Rasch model, and strict unidimensionality, SWEMWBS provides an interval scale estimate of mental well-being. Conclusion: A short 7 item version of WEMWBS was found to satisfy the strict unidimensionality expectations of the Rasch model, and be largely free of bias. This scale, SWEMWBS, provides a raw score-interval scale transformation for use in parametric procedures. In terms of face validity, SWEMWBS presents a more restricted view of mental well-being than the 14 item WEMWBS, with most items representing aspects of psychological and eudemonic well-being, and few covering hedonic well-being or affect. However, robust measurement properties combined with brevity make SWEMWBS preferable to WEMWBS at present for monitoring mental well-being in populations. Where face validity is an issue there remain arguments for continuing to collect data on the full 14 item WEMWBS

An evaluation of the structural validity of the Shoulder Pain and Disability Index (SPADI) using the Rasch model

Purpose: The Shoulder Pain and Disability Index (SPADI) has been extensively evaluated for its psychometric properties using classic test theory (CTT). The purpose of this study was to evaluate its structural validity using Rasch model analysis. Methods: Responses to the SPADI from 1030 patients referred for physiotherapy with shoulder pain and enrolled in a prospective cohort study were available for Rasch model analysis. Overall fit, individual person and item fit, response format, dependence, unidimensionality, targeting, reliability and differential item functioning (DIF) were examined. Results: The SPADI pain subscale initially demonstrated a misfit due to DIF by age and gender. After iterative analysis it showed good fit to the Rasch model with acceptable targeting and unidimensionality (overall fit (chi-square statistic 57.2, p=0.1); mean item fit residual 0.19 (1.5) and mean person fit residual 0.44 (1.1); person separation index (PSI) of 0.83). The disability subscale however shows significant misfit due to uniform DIF even after iterative analyses were used to explore different solutions to the sources of misfit (overall fit (chi-square statistic 57.2, p=0.1); mean item fit residual -0.54 (1.26) and mean person fit residual -0.38 (1.0); PSI 0.84). Conclusions: Rasch Model analysis of the SPADI has identified some strengths and limitations not previously observed using CTT methods. The SPADI should be treated as two separate subscales. The SPADI is a widely used outcome measure in clinical practice and research, however the scores derived from it must be interpreted with caution. The pain subscale fits the Rasch model expectations well. The disability subscale does not fit the Rasch model and its current format does not meet the criteria for true interval-level measurement required for use as a primary endpoint in clinical trials. Clinicians should therefore exercise caution when interpreting score changes on the disability subscale and attempt to compare their scores to age and sex stratified data